Automatic Annotation Techniques for Supervised and Semi-supervised Query-focused Summarization

In this paper, we study one semi-supervised and several supervised methods for extractive query-focused multi-document summarization. Traditional approaches to multidocument summarization are either unsupervised or supervised. The unsupervised approaches use heuristic rules to select the most important sentences, which are hard to generalize. On the other hand, huge amount of annotated data is a prerequisite for supervised training, the availability of which is very rare for a very new research problem like query-focused summarization. However, the availability of the abstract summaries from different evaluation framework allows us to experiment with the semi-supervised approach and the sentence alignment methods to annotate the document sentences automatically. We employ five different automatic annotation techniques to build the extracts from the human abstracts. We use TF*IDF1 based cosine similarity, Extended String Subsequence Kernel (ESSK), Basic Element (BE) overlap, Syntactic Similarity, and Semantic Similarity measures as the annotation methods. Based on these annotations, we experiment with: a) two supervised multi-class classifiers; Support Vector Machines (SVM) and Logistic Regression (LR), b) three regression models; SVM, Bagging and Gaussian Processes (GP), and c) one sequence labeler; Conditional Random Fields (CRF). Our initial results of SVM classifier based on a very small subset of DUC-2006 and DUC-2007 data show the effectiveness of our approaches. TF=Term Frequency, IDF=Inverse Document Frequency